\chapter{SVMBR program overview}
\label{SVMBR-PROGRAMA}

\section{Introduction}

Although several programs to training SVMs may
be found over the Internet, the main reason to develop a
new one was to create a framework where not only new kernels
could be added, as happens with the majority of SVMs programs, but
also it was allowed to add new training algorithms and QP
solver strategies.

SVMBR was developed from scratch, in C++, after a careful class
modeling and can be compiled to run on any operational 
system with a standard C++ compiler. 
At this moment, SMO and EDR are the only training methods
available and there are four kernel functions supported:
linear, polynomial, RBF and sigmoid.

\section{Internal structure}

SVMBR is constituted of six main portions of code, called packages
(Figure \ref{FIG-UML-PACKAGES}):
%--
\begin{description}
	\item[Data package:] Responsible for dealing with data sets 
	  and to calculate internal products. Fast internal products
	  for sparse (with many zeros) and binary (with only 0 and 1)
	  data sets are supported (Figure \ref{FIG-UML-DATA}).
	\item[Kernel package:] Performs kernel functions evaluations.
		Linear, polynomial, RBF and sigmoid kernels are supported at
		this moment but other types can be easily added
		(Figure \ref{FIG-UML-KERNEL}).
	\item[QP Solver package:] Methods to solve the QP problem for
		SVMs are implemented in this module. SMO and EDR are working
		and a new method based on Boosting is under development
		(Figure \ref{FIG-UML-SOLVER}).
	\item[Utility package:]	Provides classes for cache, timing and
		chunk implementation.
	\item[Types packages:]	Responsible for global structures, 
		definitions and	data types.
	\item[LineArgs package:] Auxiliary class to interprets the command 
	  line arguments or a	file with all options inside it.
	\item[SVM package:]	Use all previous packages to training and
	  testing SVMs (Figure \ref{FIG-UML-SVM}).
	  %Multi-classification is supported but on a 
	  %two-by-two training basis.
\end{description}
%--
\section{Using SVMBR}

SVMBR can be used specifying all options in the command line
or using a configuration file. Configuration file is more
flexible and the unique way to run multi classification.
The configuration file is divided into sections, called 
{\ttfamily [GLOBAL]} and {\ttfamily [CLASS:x]}, where
{\ttfamily x} is a number starting from zero 
that designates the class under configuration (targets must
follow this rule as well).
Some parameters are accepted only inside {\ttfamily [GLOBAL]}
section and other can be placed in both sections (See 
Section \ref{SEC-SVM-USAGE} for details).

Data set files starts with two numbers for specifying the number of 
training vectors and the input vector dimension, respectively. 
After these two numbers, other values are considered as    
data. It is expected one data vector per row followed by its target. 
Numbers must be   separated by spaces. For instance, dataset
for XOR problem is stated as (please, remove comments):                         

{
\footnotesize
\begin{verbatim}
    4        // number of training vectors      
    2        // input dimension                
    0  0 -1  // vector 1 + target              
    0  1  1  // vector 2 + target              
    1  0  1  // vector 3 + target              
    1  1 -1  // vector 4 + target              
\end{verbatim}
}
Some examples:
%--
\begin{enumerate}
	\item Only training:
	{
	\footnotesize
	\begin{verbatim}
svmbr -tp train.dat -sv svm.net -solver smo -k rbf -p1 1.2 -p2 0.3 -c 2 
	\end{verbatim}
	}
	\item Only testing, using a trained SVM:
	{
	\footnotesize
	\begin{verbatim}
svmbr -gp test.dat -sv svm.net -go result.txt  
	\end{verbatim}
	}
	\item Training and testing:
	{
	\footnotesize
  \begin{verbatim}
svmbr -tp train.dat -gp test.dat -sv svm.net -go result.txt 
      -solver smo -k rbf -p1 1.2 -p2 0.3 -c 2 
	\end{verbatim}	
	}
	\item Training and testing but using a configuration file:
	{
	\footnotesize
  \begin{verbatim}
svmbr config.txt
	\end{verbatim}	
	}	
	config.txt contents:
	{
	\footnotesize
  \begin{verbatim}
[GLOBAL]
;
; files
;
DATAREPRESENTATION=sparse
TRAININGPATTERNFILE=train.dat
TESTINGPATTERNFILE=test.dat
TESTINGOUTFILE=result.txt
SVMFILE=svm.net
;
; kernel
;
KERNELTYPE=poly
P1=2
P2=1
P3=10
;
; optimization parameters
;
C=5.3
TOL=0.0001
EPS=0.0001
SOLUTIONMETHOD=edr
NUMEDR=4
EDRPOWER=2
	\end{verbatim}	
	}	
	\item Multi classification via configuration file:
	{
	\footnotesize
  \begin{verbatim}
svmbr config.txt
	\end{verbatim}	
	}	
	config.txt contents:
	{
	\footnotesize
  \begin{verbatim}
[GLOBAL]
; Parameters used in all classes except when 
; explicitly overwritten in a class section [CLASS:x]
NUMCLASSES=4
TRAININGPATTERNFILE=train.dat
TESTINGPATTERNFILE=test.dat
TESTINGOUTFILE=result.txt
KERNELTYPE=rbf
P1=1.23
P2=0.85
C=1.5
TOL=0.0001
EPS=0.0001
SOLUTIONMETHOD=smo
;
[CLASS:0]
; trained with smo
SVMFILE=svm-0.net
;
[CLASS:1]
; trained with EDR
SVMFILE=svm-1.net
SOLUTIONMETHOD=edr
NUMEDR=4
EDRPOWER=2
;
[CLASS:2]
; trained with smo and polynomial kernel
KERNELTYPE=poly
P1=5
P2=2
P3=20
C=4
SVMFILE=svm-2.net
;
[CLASS:3]
; trained with smo and other C
C=3
SVMFILE=svm-3.net
	\end{verbatim}	
	}		
\end{enumerate}
\vfill
\begin{flushright}
{\footnotesize\ttfamily 
Version: 2.0.2\\
Last update: 07/Dec/2002\\
Author: Marcelo Barros (\url{barros@smar.com.br	})
}
\end{flushright}

%--
%\begin{verbatim}
%SVM      *svm; 
%Data     *data;
%Kernel   *kernel;
%Solver   *solver;
%data = new NormalData(``data_filename'');
%svm = new TrainSVM();
%svm->setData(data);
%kernel = new RBFKernel(p1,p2,p3,data);
%svm->setKernel(kernel);
%solver = new EDRSolver(C,tolerance,epsilon,numEDR,EDRPower,svm,kernel,data,chunkSize); 
%svm->setSolver(solver);
%svm->setsvmFile(``svm_structure_filename'');
%svm->run();
%svm->printSummary();
%delete kernel;
%delete solver;
%delete svm;
%delete data;
%svm = new %TestSVM(``svm_structure_filename',``data_test_filename'',``test_result_filename'',limit);
%svm->run();
%svm->printSummary();
%delete svm;
%\end{verbatim}

\pagebreak
\section{SVMBR use summary}
\label{SEC-SVM-USAGE}
{ \footnotesize
\begin{verbatim}
Usage:(1 of 2)

svmbr -option1 value1 -option2 value2 ...

Options (type in lower case !):

KERNELS:                                                                     
=======                                                                      
-k [poly|rbf|perceptron|linear]  | Chooses the SVM kernel.                   
                                 | Kernel expressions:                       
                                 | rbf       : exp(-|x1-x2|^2/(2*p3*p1^2))   
                                 | poly      : ((x1*x2+p2)/p3)^p1            
                                 | perceptron: tanh(p1*x1*x2/p3+p2)          
                                 | linear    : (x1*x2+p2)/p3                 
                                 |                                           
-p1 value                        | Kernel parameters.                        
-p2 value                        | For RBF, p1 is the variance and p2        
-p3 value                        | is not used. For polynomial p1 is the     
                                 | degree, p2 is bias. Perceptron has p1 as  
                                 | slope and p2 as offset. Linear uses p2 as 
                                 | bias. Scale parameter is provided for     
                                 | all kernels by p3 parameter.              
                                 | Default: p1=1, p2=0, p3=1.                
                                 |                                           
FILES:                                                                       
=====                                                                        
-tp filename                     | Training patterns.                        
                                 | First two numbers specifies the number of 
                                 | training vectors and the input vector,    
                                 | dimension, respectively. After these two     
                                 | values, other points are considered as    
                                 | data. Put one data vector per row,        
                                 | followed by its target. Numbers must be   
                                 | separated by spaces. For instance, dataset
                                 | for XOR problem is stated as (please,     
                                 | remove comments):                         
                                 | ----------------------------------------  
                                 | 4       // number of training vector      
                                 | 2       // input dimension                
                                 | 0 0 -1  // vector 1 + target              
                                 | 0 1  1  // vector 2 + target              
                                 | 1 0  1  // vector 3 + target              
                                 | 1 1 -1  // vector 4 + target              
                                 | ----------------------------------------  
                                 |                                           
-gp filename                     | Testing patterns (generalization).        
                                 | First two numbers specifies the number of 
                                 | testing vectors and the input vector,     
                                 | dimension, respectively. After these two     
                                 | values, other points are considered as    
                                 | data. Put one data vector per row,        
                                 | followed by its target. Numbers must be   
                                 | separated by spaces. For instance, dataset
                                 | for XOR problem is stated as (please,     
                                 | remove comments):                         
                                 | ----------------------------------------  
                                 | 4       // number of training vector      
                                 | 2       // input dimension                
                                 | 0 0 -1  // vector 1 + target              
                                 | 0 1  1  // vector 2 + target              
                                 | 1 0  1  // vector 3 + target              
                                 | 1 1 -1  // vector 4 + target              
                                 | ----------------------------------------  
                                 |                                           
-go filename                     | Testing output file.                      
                                 | Generalization results are saved on this  
                                 | file, like SVM's output and MSE.          
                                 |                                           
-sv filename                     | SVM configuration.                        
                                 | This file describes the SVM structure. It 
                                 | is the output file from SVMBR.            
                                 |                                           
-d [normal|sparse|binary]        | Data representation.                      
                                 | This parameter affects how the vectors are
                                 | treated by the program. Set sparse when   
                                 | using training vector built mainly by     
                                 | zeros and binary when input vectors have  
                                 | only 0 and 1 values.                      
                                 | Default: d=normal                         
                                 |                                           
OPTIMIZATION:                                                                
============                                                                 
-solver [edr|smo|boosting]       | Method used to find the solution.         
                                 | Default: solver = smo                     
                                 |                                           
-c value                         | Upper limit for Lagrange multipliers.     
                                 | Default: c=1                              
                                 |                                           
-t value                         | Tolerance for KKT condition.              
                                 | Default: t=0.001                          
                                 |                                           
-e value                         | Epsilon for bound precision.              
                                 | Default: e=0.001                          
                                 |                                           
-edr value                       | Error dependent repetition.               
                                 | Number of scans performed on the training 
                                 | set.                                      
                                 | Default: edr = -1 (off)                   
                                 |                                           
-power value                     | Power for comparison function in EDR.    
                                 | (err_i)^power > j*err_max/edr.            
                                 | Default: power = 1                        
                                 |                                           
-maxiter value                   | Maximum number of iterations of SMO when  
                                 | using together with EDR.                  
                                 | Default: maxiter = -1                     
                                 |                                           
-chunk value                     | Chunking size.                            
                                 | Number of vectors in chunk.      
                                 | Default: chunk = -1 (not using chunk).  
                                 |                                           
OTHERS:                                                                      
======                                                                       
-l [0|1]                         | Hard (1) or soft limit (0) when           
                                 | evaluating output.                        
                                 | Default: l=0                              
                                 |                                           
-version                         | Print the current version.                
                                 |                                           
-h                               | Print this help message.                  
                                 |                                           

Usage:(2 of 2)

svmbr file_with_all_parameters

Options (type in upper case !):

[GLOBAL]                                                                     
NUMCLASSES=value                 | Number of classes.                        
                                 | Default: 2                                
                                 |                                           
DATAREPRESENTATION=              | Data representation.                      
[normal|sparse|binary]           | This parameter affects how the vectors are
                                 | treated by the program. Set sparse when   
                                 | using training vector built mainly by     
                                 | zeros and binary when input vectors have  
                                 | only 0 and 1 values.                      
                                 | Default: d=normal                         
                                 |                                           
SOLUTIONMETHOD=[edr|smo|boosting]| Method used to find the solution.         
                                 | Default: smo                              
                                 |                                           
CHUNKSIZE=value                  | Chunking size.                            
                                 | Number of vectors in chunk.      
                                 | Default: chunk = -1 (not using chunk).  
                                 |                                           
TRAININGPATTERNFILE=filename     | Training patterns.                        
                                 | First two numbers specifies the number of 
                                 | training vectors and the input vector,    
                                 | dimension, respectively. After these two     
                                 | values, other points are considered as    
                                 | data. Put one data vector per row,        
                                 | followed by its target. Numbers must be   
                                 | separated by spaces. For instance, dataset
                                 | for XOR problem is stated as (please,     
                                 | remove comments):                         
                                 | ----------------------------------------  
                                 | 4       // number of training vector      
                                 | 2       // input dimension                
                                 | 0 0 -1  // vector 1 + target              
                                 | 0 1  1  // vector 2 + target              
                                 | 1 0  1  // vector 3 + target              
                                 | 1 1 -1  // vector 4 + target              
                                 | ----------------------------------------  
                                 | For multi class problems, the targets are 
                                 | represented by 0,1,2,3... For instance:   
                                 | 6       // number of training vector      
                                 | 2       // input dimension                
                                 | 0 7 0   // vector 1 + target              
                                 | 2 1 0   // vector 2 + target              
                                 | 4 4 1   // vector 3 + target              
                                 | 3 1 2   // vector 4 + target              
                                 | 2 4 0   // vector 5 + target              
                                 | 7 6 1   // vector 6 + target              
                                 |                                           
TESTINGPATTERNFILE=filename      | Testing patterns (generalization).        
                                 | First two numbers specifies the number of 
                                 | testing vectors and the input vector,     
                                 | dimension, respectively. After these two     
                                 | values, other points are considered as    
                                 | data. Put one data vector per row,        
                                 | followed by its target. Numbers must be   
                                 | separated by spaces. For instance, dataset
                                 | for XOR problem is stated as (please,     
                                 | remove comments):                         
                                 | ----------------------------------------  
                                 | 4       // number of training vector      
                                 | 2       // input dimension                
                                 | 0 0 -1  // vector 1 + target              
                                 | 0 1  1  // vector 2 + target              
                                 | 1 0  1  // vector 3 + target              
                                 | 1 1 -1  // vector 4 + target              
                                 | ----------------------------------------  
                                 | For multi class problems, the targets are 
                                 | represented by 0,1,2,3... For instance:   
                                 | 6       // number of training vector      
                                 | 2       // input dimension                
                                 | 0 7 0   // vector 1 + target              
                                 | 2 1 0   // vector 2 + target              
                                 | 4 4 1   // vector 3 + target              
                                 | 3 1 2   // vector 4 + target              
                                 | 2 4 0   // vector 5 + target              
                                 | 7 6 1   // vector 6 + target              
                                 |                                           
TESTINGOUTFILE=filename          | Testing output file.                      
                                 | Generalization results are saved on this  
                                 | file, like SVM's output and MSE.          
                                 |                                           
                                                                             
[GLOBAL] or [CLASS:x]                                                        
(x is the number of the class: 0,1,2,3...)                                   
                                                                             
SVMFILE=filename                 | SVM configuration.                        
                                 | This file describes the SVM structure. It 
                                 | is the output file from SVMBR             
                                 | Only for 2 classes it is global, if not   
                                 | it is not global, it has to exist in each 
                                 | class parameters.                         
                                                                             
[CLASS:x]                                                                    
(x is the number of the class: 0,1,2,3...)                                   
                                                                             
KERNELTYPE=                      | Chooses the SVM kernel.                   
[poly|rbf|perceptron|linear]     | Kernel expressions:                       
                                 | rbf       : exp(-|x1-x2|^2/(2*p3*p1^2))   
                                 | poly      : ((x1*x2+p2)/p3)^p1            
                                 | perceptron: tanh(p1*x1*x2/p3+p2)          
                                 | linear    : (x1*x2+p2)/p3                 
                                 |                                           
P1=value                         | Kernel parameters.                        
P2=value                         | For RBF, p1 is the variance and p2        
P3=value                         | is not used. For polynomial p1 is the     
                                 | degree, p2 is bias. Perceptron has p1 as  
                                 | slope and p2 as offset. Linear uses p2 as 
                                 | bias. Scale parameter is provided for     
                                 | all kernels by p3 parameter.              
                                 | Default: P1=1, P2=0, P3=1.                
                                 |                                           
                                 |                                           
C=value                          | Upper limit for Lagrange multipliers.     
                                 | Default: 1                                
                                 |                                           
TOL=value                        | Tolerance for KKT condition.              
                                 | Default: 0.001                            
                                 |                                           
EPS=value                        | Epsilon for bound precision.              
                                 | Default: 0.001                            
                                 |                                           
NUMEDR=value                     | Error dependent repetition.               
                                 | Number of scans performed on the training 
                                 | set.                                      
                                 | Default: -1 (off)                         
                                 |                                           
EDRPOWER=value                   | Power for comparison function in EDR.    
                                 | (err_i)^power > j*err_max/edr.            
                                 | Default: power = 2                        
                                 |                                           
MAXITER=value                    | Maximum number of iterations of SMO when  
                                 | using together with EDR.                  
                                 | Default: -1                               
                                 |                                           
LIMIT=[0|1]                      | Hard (1) or soft limit (0) when           
                                 | evaluating output.                        
                                 | Default: l=0                              
                                 |                                           
 It is important to observe these rules when using the parameters file
 - file cannot have spaces before any word
 - file cannot have spaces after '='
 - file must have return in the last line
 - file can have lines for comment  using # or ; at beginning
\end{verbatim}
} % end footnotesize

\pagebreak
% Para que as figuras do uml saissem tive que configurar
% a impressora para o maximo de compatibilidade e eps
% -------------
\begin{figure}
\centering
\includegraphics[scale=.8,trim= 20 550 150 0]{figs/umlpackages.eps}
\caption{\label{FIG-UML-PACKAGES}SVMBR packages.}
\end{figure}
% -------------

\begin{figure}
\centering
\includegraphics[scale=.8,trim= 20 370 200 0]{figs/umldata.eps}
\caption{\label{FIG-UML-DATA}Data package class structure.}
\end{figure}
% -------------
\begin{figure}
\centering
\includegraphics[scale=.8,trim= 20 450 100 0]{figs/umlkernel.eps}
\caption{\label{FIG-UML-KERNEL}Kernel package class structure.}
\end{figure}
% -------------
\begin{figure}
\centering
\includegraphics[scale=.8,trim= 20 250 150 0]{figs/umlsolver.eps}
\caption{\label{FIG-UML-SOLVER}Solver package class structure.}
\end{figure}
% -------------
\begin{figure}
\centering
\includegraphics[scale=.8,trim= 20 350 100 0]{figs/umlsvm.eps}
\caption{\label{FIG-UML-SVM}SVM package class structure.}
\end{figure}
% -------------


